Hard Pixel Mining for Depth Privileged Semantic Segmentation

Semantic segmentation has achieved remarkable progress but remains challenging due to the complex scene, object occlusion, and so on. Some research works have attempted use extra information such as a depth map help RGB based semantic because could provide complementary geometric cues. However, inaccessibility of sensors, is usually unavailable for test images. In this paper, we leverage only training images privileged mine hard pixels in segmentation, which available not Specifically, propose novel Loss Weight Module, outputs loss weight by employing two depth-related measurements pixels: Depth Prediction Error Depth-aware Segmentation Error. The then applied loss, with goal learning more robust model paying attention pixels. Besides, also explore curriculum strategy on map. Meanwhile, fully different scales, apply our module multi-scale side outputs. Our mining method achieves state-of-the-art results three benchmark datasets, even outperforms methods need input during testing.